Electronic device, method of controlling the same, and storage medium

ABSTRACT

An electronic device comprises a display configured to display an image, an ocular optical system for viewing the display, and a control unit configured to control the display unit so as to change a display area of the display based on a distance from the ocular optical system to an eye of a user looking into the ocular optical system.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a technique for changing a display areaof a display provided in an electronic device.

Description of the Related Art

Some models of image capturing devices, such as a digital camera and avideo camera, use an electronic viewfinder (EVF) for visually confirminga subject. The electronic viewfinder is configured so that a smalldisplay provided inside the camera can be viewed magnified through anocular optical system configured by a plurality of lenses. A user canview a magnified display image by looking into this ocular opticalsystem.

Recently, there has been a trend of desiring a higher displaymagnification for viewfinders of cameras. The higher the magnification,the bigger the image can be seen, so it has the merit that it is easierto confirm focus. In addition, the user feels more immersed with aviewfinder with a large field of view, which makes it more fun tocapture images.

However, when a distance from the viewfinder to the eye increases (e.g.,when the user looks into the viewfinder while wearing glasses, and thelike), if the display area is too large, there will be a problem that aportion of the display area will be shielded, resulting in poorervisibility and making framing more difficult.

As one countermeasure for this problem, Japanese Patent Laid-Open No.2010-016669 discloses a technique that allows a user to arbitrarily setthe display area of the viewfinder.

However, in the prior art disclosed in Japanese Patent Laid-Open No.2010-01666, when changing the display area of the viewfinder, it isnecessary to operate from a hierarchical menu. Therefore, there is aproblem that an operation for changing the display area is cumbersomefor users who, at times, capture images while wearing glasses and, atother times, capture images without wearing glasses.

SUMMARY OF THE INVENTION

The present invention has been made in view of the problems describedabove and provides an electronic device capable of displaying an imagein a display area of an appropriate size for a user.

According to a first aspect of the present invention, there is providedan electronic device comprising: a display configured to display animage; an ocular optical system configured to view the display; and atleast one processor or circuit configured to function as a control unitconfigured to control the display so as to change a display area of thedisplay based on a distance from the ocular optical system to an eye ofa user looking into the ocular optical system.

According to a second aspect of the present invention, there is provideda method of controlling an electronic device including a displayconfigured to display an image and an ocular optical system configuredto view the display, the method comprising: controlling the display soas to change a display area of the display based on a distance from theocular optical system to an eye of a user looking into the ocularoptical system.

According to a third aspect of the present invention, there is provideda non-transitory computer-readable storage medium storing a program forcausing a computer to execute the method of controlling an electronicdevice including a display configured to display an image and an ocularoptical system configured to view the display, the method comprising:controlling the display so as to change a display area of the displaybased on a distance from the ocular optical system to an eye of a userlooking into the ocular optical system.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are diagrams illustrating an external appearance of adigital, interchangeable-lens camera, which is a first embodiment of anelectronic device of the present invention.

FIG. 2 is a cross-sectional view of the camera of the first embodiment.

FIG. 3 is a cross-sectional view of an optical system that includes aline-of-sight detection mechanism.

FIGS. 4A and 4B are perspective views of the optical system thatincludes the line-of-sight detection mechanism.

FIG. 5 is an optical path diagram for when detecting a line of sightusing the line-of-sight detection mechanism.

FIG. 6 is a schematic diagram for explaining principles of a method ofdetecting a field of view.

FIGS. 7A and 7B are schematic diagrams illustrating an image of an eye.

FIG. 8 is a flowchart for explaining an operation for detecting a lineof sight.

FIG. 9 is a schematic diagram illustrating a distance from the finalsurface of an ocular optical system to an eye.

FIGS. 10A and 10B are schematic diagrams illustrating how to change adisplay area of a display device.

FIGS. 11A and 11B are schematic diagrams illustrating changing a displayof targets used for calibration based on the distance from the finalsurface of the ocular optical system to an eye.

FIG. 12 is a flowchart for explaining an operation for changing thedisplay area of the display device.

FIGS. 13A and 13B are schematic diagrams illustrating how to change thedisplay area of the display device in a second embodiment.

FIG. 14 is a flowchart for explaining an operation for changing thedisplay area of the display device in a third embodiment.

FIG. 15 is a flowchart for explaining an operation for changing thedisplay area of the display device in a fourth embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention. Multiple features aredescribed in the embodiments, but limitation is not made to an inventionthat requires all such features, and multiple such features may becombined as appropriate. Furthermore, in the attached drawings, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

First Embodiment

<Description of Configuration>

FIGS. 1A and 1B are diagrams illustrating an external appearance of adigital, interchangeable-lens camera 1 (hereinafter, the camera), whichis a first embodiment of an electronic device of the present invention.The electronic device referred to in the present invention is notlimited to a digital camera and encompasses devices for displayinginformation, such as images and text, and any electronic device capableof detecting a line-of-sight of a user viewing an optical image throughan ocular optical system. These electronic devices may include, forexample, mobile phones, game machines, tablet terminals, personalcomputers, watch and eyewear-type information terminals, head-mounteddisplays, binoculars, and the like.

FIG. 1A is a front perspective view of the camera 1, and FIG. 1B is arear perspective view of the camera 1.

As illustrated in FIG. 1A, the camera 1 includes an image capturing lensunit 1A and a camera body 1B. The camera body 1B is provided with arelease button 5, which is an operation member for receiving an imagecapturing operation from a user (photographer). As illustrated in FIG.1B, the back of the camera body 1B is provided with an ocular windowframe 121, which the user looks through to view a display device 6,which is included in the camera body 1B and will be described later (seeFIG. 3 ).

A display unit in the present embodiment includes the display device 6.The ocular window frame 121 forms a viewing port 13 and protrudesoutward (from the back side) with respect to the camera body 1B. Theback of the camera body 1B is also provided with operation members 41 to43 for receiving various operations from the user. For example, theoperation member 41 is a touch panel that accepts a touch operation ofthe user, the operation member 42 is an operation lever that can bepushed down in respective directions, and the operation member 43 is afour-direction key that can be pushed in each of the four directions.The operation member 41 (touch panel) is provided with a display panel40, such as a liquid crystal panel (see FIG. 3 ), and includes afunction of displaying an image.

FIG. 2 is a cross-sectional side view of the camera 1 of the presentembodiment and is a diagram illustrating an electrical blockconfiguration in the camera 1. The camera body 1B includes an imagecapturing element 2 for capturing a subject image. The image capturingelement 2 is an image capturing element configured by, for example, aCCD or CMOS sensor or the like; an optical image formed on an imagecapturing plane of the image capturing element 2 by an optical system ofthe image capturing lens unit 1A is photoelectrically converted, and anobtained analog image signal is A/D-converted and outputted as imagedata.

The image capturing lens unit 1A is configured to include an opticalsystem that includes a zoom lens, a focus lens, a diaphragm, and thelike and, when mounted on the camera body 1B, guides a light beam from asubject to the image capturing element 2, thereby forming an image ofthe subject on an image capturing plane of the image capturing element2. A diaphragm control unit 118, a focus adjustment unit 119, and a zoomcontrol unit 120 each receive an instruction signal from a CPU 3 througha mount contact unit 117 and drive and control the diaphragm, the focuslens, and the zoom lens, respectively, in accordance with theinstruction signal.

The CPU 3 provided in the camera body 1B reads control programs for therespective blocks provided in the camera body 1B from a ROM included ina memory unit 4, deploys these in a RAM included in the memory unit 4,and executes these. Thus, the CPU 3 controls the operation of therespective blocks provided in the camera body 1B. The CPU 3 is connectedto a line-of-sight detection unit 201, a photometry unit 202, anautofocus detection unit 203, a signal input unit 204, an eye-approachdetection unit 208, a distance calculation unit 209, a display devicedriving unit 210, a light source driving unit 205, and the like.Further, the CPU 3 transmits a signal to the diaphragm control unit 118,the focus adjustment unit 119, and the zoom control unit 120 provided inthe image capturing lens unit 1A through a mount contact point 117. Inthe present embodiment, the memory unit 4 includes a function of storingimage capturing signals from the image capturing element 2 and aline-of-sight detection sensor 30.

In a state in which an image of an eyeball is formed on theline-of-sight detection sensor 30 (CCD-EYE), the line-of-sight detectionunit 201 performs A/D conversion of an output (an image of an eye inwhich the eyeball is captured) of the line-of-sight detection sensor 30and transmits that result to the CPU 3. The CPU 3 extracts featurepoints necessary for detecting a line of sight from the image of the eyein accordance with a predetermined algorithm, which will be describedlater, and calculates a line of sight (a gaze point in an image forvisual confirmation) of the user from positions of the feature points.

Further, the distance calculation unit 209 calculates a distance 32 fromthe final surface of an ocular optical system 16, which is illustratedin FIG. 9 , to the eye. The distance calculation unit 209 calculates thedistance 32 from the final surface of the ocular optical system 16 tothe eye based on coordinates of cornea-reflected images on theline-of-sight detection sensor 30 and transmits an output value to theCPU 3. The distance calculation unit 209 may obtain the distance 32 froma table or the like in which the distance 32 from the final surface ofthe ocular optical system 16 to the eye has been calculated in advancebased on the coordinates of the cornea-reflected images.

The display device driving unit 210 determines the display area of thedisplay device 6 based on the distance calculated by the distancecalculation unit 209 and performs display. The eye-approach detectionunit 208 transmits an output of an eye-approach detection sensor 50 tothe CPU 3. The CPU 3 calculates whether or not the user's eye hasapproached in accordance with a predetermined algorithm, which will bedescribed later. The light source driving unit 205 drives infrared LEDs18 to 27, which are light sources, to be at a predetermined emissionintensity in accordance with commands from the CPU 3.

The photometry unit 202 performs amplification, logarithmic compression,A/D conversion and the like of a signal (more specifically, a luminancesignal, which corresponds to the brightness of a field of view) obtainedfrom the image capturing element 2, which also serves as a photometricsensor, and transmits that result to the CPU 3 as field-of-viewluminance information.

The autofocus detection unit 203 A/D-converts signal voltages from aplurality of detection elements (a plurality of sub-pixels), which areincluded in the pixels in the image capturing element 2 and are used fordetecting a phase difference, and transmits them to the CPU 3. The CPU 3calculates, from signals of the plurality of detection elements, adistance to a subject corresponding to each focus detection point. Thisis a known technique, which is known as image plane phase difference AF.In the present embodiment, as an example, it is assumed that an image ofa field of view (image for visual confirmation) in the viewfinder isdivided and that there is a focus detection point at each of 180locations, which have been obtained by division, on the image capturingplane.

The image processing unit 206 performs various kinds of image processingon image data stored in the RAM in the memory unit 4. Specifically,various image processes for developing, displaying, and recordingdigital image data are performed, such as defect correction processingof pixels caused by an optical system or an image capturing element,demosaicing processing, white balance correction processing, colorinterpolation processing, and gamma processing, for example.

A switch SW1, which turns ON at a first stroke of the release button 5and is for starting the camera 1's photometry, focus detection, andline-of-sight detection operations and the like, and a switch SW2, whichturns ON at a second stroke of the release button 5 and is for startingan image capturing operation are connected to the signal input unit 204.An ON signal is inputted from the switch SW1 or SW2 to the signal inputunit 204 and is then transmitted to the CPU 3. The signal input unit 204also receives operation inputs from the operation members 41 (a touchpanel), 42 (a button), and 43 (arrow keys) illustrated in FIG. 1B.

A recording/output unit 207 records data, which includes image data, ona storage medium, such as a removable memory card, or outputs the datato an external device via an external interface.

FIG. 3 is a cross-sectional view of an optical system that includes aline-of-sight detection mechanism of the present embodiment and is adiagram obtained by cutting the camera 1 across a YZ plane, which isformed by a Y-axis and a Z-axis illustrated in FIG. 1A.

A shutter 44 and the image capturing element 2 are arranged in order inan optical axis direction of the image capturing lens unit 1A. The backof the camera body 1B is provided with the display panel 40, which isused for operating the camera 1, displaying menus for viewing andediting images obtained with the camera 1, and displaying images. Thedisplay panel 40 is configured by a liquid crystal panel with abacklight, an organic EL panel, or the like.

A panel holder 7 is a panel holder for holding the display device 6configured by an organic EL panel or the like, is bonded and fixed tothe display device 6, and configures a display panel unit 8.

A first optical path splitting prism 9 and a second optical pathsplitting prism 10 are bonded by adhesion to configure an optical pathsplitting prism unit 11 (optical path splitting member). The opticalpath splitting prism unit 11 guides a light beam from the display device6 to an eyepiece window 17 provided in the user's viewing port 13 and,conversely, guides, to the line-of-sight detection sensor 30 illustratedin FIGS. 4A and 4B, light reflected from an eye (pupil) and the like andguided from the eyepiece window 17. The display panel unit 8 and theoptical path splitting prism unit 11 are fixed with a mask 12therebetween and are formed into a unit.

The ocular optical system 16 is configured by a G1 lens 13, a G2 lens14, and a G3 lens 15. The electronic viewfinder is configured such thatthe display panel unit 8 is seen magnified through the ocular opticalsystem 16 so that the user can view a magnified display image.

The eyepiece window 17 is a transparent member that transmits visiblelight. An image displayed on the display panel unit 8 is viewed throughthe optical path splitting prism unit 11, the ocular optical system 16,and the eyepiece window 17.

Illumination windows 20 and 21 are windows for hiding the infrared LEDs18 and 19 so as not to be visible from the outside and are configured byresin that absorbs visible light and transmits infrared light.

In addition to being able to display menus and images similarly to thedisplay panel 40 as a typical electronic viewfinder (EVF), the EVFprovided in the camera body 1B in the present embodiment is configuredso as to be able to detect a line of sight of the user looking at theEVF and reflect a result of the detection in the control of the camera1.

Similarly to the display panel 40, when the user is looking through theviewfinder, the display device 6 is used to display menus for operatingthe camera 1 and viewing and editing images obtained by the camera 1 anddisplay images. The display device 6 is configured by a liquid crystalpanel with a backlight, an organic EL panel, or the like.

FIGS. 4A and 4B are a perspective view and a cross-sectional view of anoptical system that includes a line-of-sight detection mechanism of thepresent embodiment. FIG. 4A is a perspective view illustrating aconfiguration of the EVF in the present embodiment, and FIG. 4B is across-sectional side view of an optical axis of the EVF.

The eyepiece window 17 is a transparent member that transmits visiblelight. An image displayed on the display panel 6 is viewed through theoptical path splitting prism unit 11, the ocular optical system 16, andthe eyepiece window 17.

The infrared LEDs 18, 19, 22, 23, 24, 25, 26, and 27 are arranged so asto irradiate infrared light toward the user's viewing port 13 fromdifferent positions and orientations. The illumination windows 20 and 21are windows for hiding the infrared LEDs 18, 19, 22, 23, 24, 25, 26, and27 so as not to be visible from the outside and are configured by resinthat absorbs visible light and transmits infrared light.

The infrared LED 18, 19, 23, and 25 are infrared LEDs for close-rangeillumination. The infrared LED 22, 24, 26, and 27 are infrared LEDs forlong-range illumination. A line-of-sight detection optical system, whichincludes a diaphragm 28 and a line-of-sight image forming lens 29,further guides reflected infrared light guided from the eyepiece window17 by the optical path splitting prism unit 11 to the line-of-sightdetection sensor 30. The line-of-sight detection sensor 30 is configuredby a solid-state image capturing element, such as a CCD or CMOS sensor.

The eye-approach detection sensor 50 is configured by a photodiode,which can be driven at a power that is lower than the line-of-sightdetection sensor 30, and the like. The infrared LED 22 for detecting aline of sight is also used as an infrared LED for detecting the approachof the eye. The infrared LED 22 illuminates the user's eye, and theeye-approach detection sensor 50 receives light diffusely reflected offthe user.

In FIG. 4B, an image of an eyeball, which has been illuminated byinfrared LEDs, of the user looking through the eyepiece window 17 entersthe second optical path splitting prism 10 from a second surface 10 athrough the G3 lens 15, the G2 lens 14, and the G1 lens 13. This opticalpath is indicated by 31 a. On a first surface 10 b of the second opticalpath splitting prism is formed a dichroic film for reflecting infraredlight.

An image of an eyeball illuminated by at least one of the infrared LEDsillustrated in FIG. 4A is reflected by the first surface 10 b toward thesecond surface 10 a. This reflected light path is indicated by 31 b. Theinfrared light that has passed the reflected light path 31 b is totallyreflected by the second surface 10 a, passes an image forming light path31 c, and is formed into an image on the line-of-sight detection sensor30 by the line-of-sight image forming lens 29 through the diaphragm 28.

To detect a line of sight, a cornea-reflected image formed by a specularreflection of infrared LED by the cornea is used in conjunction with aneyeball image by illumination. FIG. 5 illustrates an example of anoptical path for light emitted from the infrared LEDs 18, 19, 23, and 25for close-range illumination from when it is specularly reflected by acornea 611 of an eyeball until it is received by the line-of-sightdetection sensor 30.

<Description of Operation for Detecting Line of Sight>

A method of detecting a line of sight will be described with referenceto FIGS. 6, 7A, 7B, and 8 .

FIG. 6 is a diagram for explaining principles of a method of detecting aline of sight and is a schematic diagram of an optical system forperforming line-of-sight detection.

As illustrated in FIG. 6 , light sources 601 a and 601 b are arranged tobe substantially symmetrical with respect to an optical axis of a lightreceiving lens 618 (corresponds to the line-of-sight image forming lens29 in FIG. 4B) and illuminates the user's eyeball 610. Some of the lightemitted from the light sources 601 a and 601 b and reflected by theeyeball 610 are focused on the line-of-sight detection sensor 620(corresponds to the line-of-sight detection sensor 30 in FIGS. 4A, 4B,and 5 ) by the light receiving lens 618.

FIG. 7A is a schematic diagram of an eye image captured by theline-of-sight detection sensor 620 (an eyeball image projected on theline-of-sight detection sensor 620), and FIG. 7B is a diagramillustrating an output intensity of the image capturing element in theline-of-sight detection sensor 620. FIG. 8 indicates a schematicflowchart of an operation for detecting a line of sight.

When an operation for detecting a line of sight is started, in step S801of FIG. 8 , the CPU 3 causes the light sources 601 a and 601 b to emitlight, irradiating infrared light of an emission intensity E2 forline-of-sight detection toward the user's eyeball 610. An image of theuser's eyeball illuminated by the infrared light is formed on theline-of-sight detection sensor 620 through the light receiving lens 618and is photoelectrically converted by the line-of-sight detection sensor620. By this, a processable electrical signal of an eye image isobtained.

In step S802, the CPU 3 obtains the eye image (an eye image signal; anelectrical signal of the eye image) from the line-of-sight detectionsensor 620 using the line-of-sight detection unit 201.

In step S803, the CPU 3 obtains coordinates of points corresponding tocornea-reflected images Pd and Pe of the light sources 601 a and 601 band a pupil center c from the eye image obtained in step S802.

The infrared light emitted from the light sources 601 a and 601 billuminates the cornea 611 of the user's eyeball 610. At this time, thecornea-reflected images Pd and Pe formed by some of the infrared lightreflected on the surface of the cornea 611 is focused by the lightreceiving lens 618 and formed into an image on the line-of-sightdetection sensor 620, thereby becoming cornea-reflected images Pd′ andPe′ in the eye image. Similarly, a light beam from edges a and b of apupil 612 is also formed into an image on the line-of-sight detectionsensor 620, thereby becoming pupil edge images a′ and b′ in the eyeimage.

FIG. 7B is a diagram illustrating luminance information (a luminancedistribution) of an area α′ in the eye image of FIG. 7A. In FIG. 7B, thehorizontal direction of the eye image is an X-axis direction and thevertical direction is a Y-axis direction and the luminance distributionin the X-axis direction is illustrated. In the present embodiment, theX-axis direction (horizontal) coordinates of the cornea-reflected imagesPd′ and Pe′ are Xd and Xe, and the X-axis direction coordinates of thepupil edge images a′ and b′ are Xa and Xb.

As illustrated in FIG. 7B, extremely high levels of luminance areobtained at the coordinates Xd and Xe of the cornea-reflected images Pd′and Pe′. Except for the coordinates Xd and Xe, extremely low levels ofluminance are obtained in an area from the coordinate Xa to thecoordinate Xb, which corresponds to an area of the pupil 612 (an area ofa pupil image obtained by a light beam from the pupil 612 being formedinto an image on the line-of-sight detection sensor 620). Then, in anarea of an iris 613, which is outside the pupil 612, (an area of an irisimage, which is outside the pupil image and is obtained by a light beamfrom the iris 613 being formed into an image), a luminance that is inthe middle of the above-mentioned two types of luminance is obtained.Specifically, in an area whose X-coordinate (coordinate in the X-axisdirection) is smaller than the coordinate Xa and an area whoseX-coordinate is larger than the coordinate Xb, a luminance that is inthe middle of the above-described two types of luminance is obtained.

It is possible to obtain from a luminance distribution such asillustrated in FIG. 7B, the X-coordinates Xd and Xe of thecornea-reflected images Pd′ and Pe′ and the X-coordinates Xa and Xb ofthe pupil edge images a′ and b′. Specifically, coordinates withextremely high levels of luminance can be obtained as the coordinates ofthe cornea-reflected images Pd′ and Pe′, and coordinates with extremelylow levels of luminance can be obtained as the coordinates of the pupiledge images a′ and b′. Further, when an angle of rotation θx of anoptical axis of the eyeball 610 with respect to an optical axis of thelight receiving lens 618 is small, the coordinate Xc of a pupil centerimage c′ (the center of the pupil image) obtained by a light beam fromthe pupil center c being formed into an image on the line-of-sightdetection sensor 30 can be expressed as Xc≈(Xa+Xb)/2. That is, it ispossible to calculate the coordinate Xc of the pupil center image c′from the X-coordinates Xa and Xb of the pupil edge images a′ and b′. Inthis way, the coordinates of the cornea-reflected images Pd′ and Pe′ andthe coordinate of the pupil center image c′ can be estimated.

In step S804, the CPU 3 calculates an image forming magnification β ofthe eyeball image. The image forming magnification β is a magnificationthat is determined by the position of the eyeball 610 with respect tothe light receiving lens 618 and can be obtained using a function of aninterval (Xd-Xe) between the cornea-reflected images Pd′ and Pe′.

In step S805, the CPU 3 calculates the angles of rotation of the opticalaxis of the eyeball 610 with respect to the optical axis of the lightreceiving lens 618. The X-coordinate of the middle point of thecornea-reflected image Pd and the cornea-reflected image Pe and theX-coordinate of a center of curvature ◯ of the cornea 611 almostcoincide. Therefore, when a standard distance from the center ofcurvature ◯ of the cornea 611 to the center c of the pupil 612 is ◯c,the angle of rotation θX of the eyeball 610 in the Z-X plane (planeperpendicular to the Y-axis) can be calculated by the following(Equation 1). An angle of rotation θy of the eyeball 610 in the Z-Yplane (plane perpendicular to the X-axis) can also be calculated by amethod that is the same as the method of calculating the angle ofrotation θx.

β×◯c×SINθX≈{(Xd+Xe)/2}−Xc   (Equation 1)

In step S806, the CPU 3 uses the angles of rotation θx and θy calculatedin step S805 to obtain (estimate) the user's gaze point (a position atthe end of the line of sight, the position at which the user is looking)in the image for visual confirmation displayed on the display device 6.If coordinates (Hx, Hy) of the gaze point are the coordinatescorresponding to the pupil center c, the coordinates (Hx, Hy) of thegaze point can be calculated by the following (Equation 2) and (Equation3).

Hx=m×(Ax×θx+Bx)   (Equation 2)

Hy=m×(Ay×θy+By)   (Equation 3)

The parameter m of (Equation 2) and (Equation 3) is a constantdetermined by the configuration of the viewfinder optical system (suchas the light receiving lens 618) of the camera 1, is a conversioncoefficient for converting the angles of rotation θx and θy to thecoordinates corresponding to the pupil center c in the image for visualconfirmation, and is assumed to be determined in advance and stored inthe memory unit 4. The parameters Ax, Bx, Ay, and By are line-of-sightcorrection parameters that correct for personal differences in the lineof sight, are obtained by performing calibration and are assumed to bestored in the memory unit 4 prior to the operation for detecting a lineof sight being started.

Calibration is the process of obtaining the user's eye features and isapplied for when calculating the coordinates of a gaze point from theangles of rotation. Parameters for correcting sensitivity and a shift inthe visual axis are calculated based on an eye image of when the userwas made to focus on a plurality of targets. The sensitivity iscorrected by the above parameters, Ax and Ay, and the shift in thevisual axis is corrected by the above parameters, Bx and By.

In step S807, the CPU 3 stores the coordinates (Hx, Hy) of the gazepoint in the memory unit 4 and terminates the operation for detecting aline of sight.

FIG. 9 is a diagram illustrating the distance 32 from the final surfaceof the ocular optical system to the eye.

The distance 32 from the final surface of the ocular optical system 16to the eye can be obtained using a function of the coordinates of thecornea-reflected images Pd′ and Pe′ or an interval between two points.This function is created based on the results of simulations or actualmeasurements on a real machine.

FIGS. 10A and 10B are diagrams illustrating a method of changing thedisplay area of the display.

The display area in the present embodiment refers to an area in whichOrganic Light Emitting Diodes (OLEDs) are actually illuminated out ofthe entire displayable area in which OLEDs are arranged. The displayarea of the display is changed based on the distance 32 from the finalsurface of the ocular optical system 16 to the user's eye.

As illustrated in FIG. 10A, the display area is increased when theuser's eye is close to the ocular optical system. FIG. 10A illustrates astate in which display is performed using the entire displayable area.In contrast, as illustrated in FIG. 10B, the display area is decreasedwhen the user's eye is far from the ocular optical system. In thepresent embodiment, it is assumed that the shorter the distance betweenthe user's eye and the ocular optical system, the larger the size of thedisplay area is set to be, and the farther the distance, the narrowerthe size is set to be. However, it may be switched such that if thedistance between the user's eye and the ocular optical system is equalto or less than a predetermined threshold, the display area will beincreased (for example, to the entire displayable area), and if thedistance is larger than the predetermined threshold, the display areawill be made smaller than when the distance is equal to or less than thepredetermined threshold. Further, a plurality of such thresholds may beprovided so as to change the display area in a stepwise manner beforeand after the thresholds.

Further, in the present embodiment, when changing the display area ofthe display, both the OSD (On Screen Display) display and the live viewdisplay are changed. It is assumed that the OSD display displaysinformation, such as camera settings at the time of image capturing,which include an aperture and a shutter speed, and the remaining amountof battery.

FIGS. 11A and 11B are diagrams illustrating a change of a display oftargets used for calibration (CAL) based on the distance 32 from thefinal surface of the ocular optical system 16 to the eye.

In the present embodiment, at the time of calibration for line-of-sightdetection, the distance 32 from the final surface of the ocular opticalsystem 16 to the eye is measured while the user is looking at a targetat the center, and calibration contents are changed based on thatdistance. In the calibration for line-of-sight detection, the user needslook at a specified plurality of targets for a fixed period of time.

As illustrated in FIG. 11A, if the user's eye is close to the ocularoptical system, the arrangement of peripheral targets is changed basedon that distance. In the calibration, the larger the image height of theperipheral targets from the optical axis of the ocular optical system16, the higher the accuracy will be. If the distance 32 from the finalsurface of the ocular optical system 16 to the eye is short, since thedisplay area that can be stably viewed by the user is broad, theperipheral targets can be arranged diagonally as illustrated in FIG.11A, which makes it possible to secure the image height of theperipheral targets.

As illustrated in FIG. 11B, if the user's eye is far from the ocularoptical system 16, the image height of the peripheral targets is changedso as to be reduced based on that distance. The display area that theuser can stably view changes depending on the measured distance 32 fromthe final surface of the ocular optical system 16 to the eye. Forexample, if the distance is shorter, the area that can be seen stablywill be broader; if the distance is longer, the area that can be seenstably will be smaller. Therefore, placing the peripheral targets inthat area makes it possible to ensure that the user looks at theperipheral targets.

FIG. 12 is a flowchart for explaining an operation for changing thedisplay area of the display device. This operation is started by theuser turning the power of the camera 1 on in step S1201.

In step S1202, the CPU 3 displays, for example, on the display panel 40,a display asking the user whether to perform calibration.

If the user answers “perform calibration”, the CPU 3 advances theprocess to step S1203 and performs calibration. At this time, thedistance 32 from final surface of the ocular optical system 16 to theeye is calculated.

In step S1204, the CPU 3 determines the user-optimized display area ofthe display based on the distance 32 from the final surface of theocular optical system 16 to the eye calculated in step S1203.

If the user answers “do not perform calibration” in step S1202, the CPU3 advances the process to step S1205. In step S1205, the CPU 3 reads outthe display area determined when the calibration was previouslyperformed. At this time, the user needs to select user-specificcalibration data.

In step S1206, the CPU 3 determines the display area based on thedisplay area read out in step S1205.

In step S1207, the CPU 3 actually changes the display area of thedisplay device 6 based on the determined display area.

As described above, in the present embodiment, in an electronic device,a display area of a display unit is changed based on a distance from thefinal surface of an ocular optical system to an eye. Specifically, thedisplay area is larger when the distance is shorter, and the displayarea is smaller when the distance is longer. Thus, without troublesomesetting or the like, it is possible to display an image in a displayarea that is optimal for the user.

Further, in the present embodiment, the distance from the final surfaceof the ocular optical system to the eye is calculated usingcornea-reflected images. It is possible to calculate the distance fromthe final surface of the ocular optical system to the eye based on thecoordinates of the cornea-reflected images of an image obtained by aline-of-sight detection sensor. Thus, as compared with a configurationin which a line-of-sight detection means and a distance calculationmeans are provided separately, since a means for calculating a distanceis not specifically needed, it is possible to prevent the device frombecoming complex and increasing in size, which makes it possible to havea more inexpensive configuration.

Further, in the present embodiment, at the time of calibration forline-of-sight detection, the display of targets used for calibration ischanged according to the distance from the final surface of the ocularoptical system to the eye. This allows for greater calibration certaintyand effectiveness compared to a configuration that does not change thedisplay of the targets used for calibration according to the distancefrom the final surface of the ocular optical system to the eye.Therefore, it is expected that stability and accuracy of line-of-sightdetection will improve.

Further, in the present embodiment, when changing the display area ofthe display, both the OSD display and the live view display are changed.Thus, visibility can be ensured not only in the live view display butalso in the OSD display as compared with a configuration in which onlythe live view display is changed and the OSD display is kept constant.Therefore, it is easier for the user to ascertain information, such ascamera settings and remaining amount of battery at the time of imagecapturing.

Further, in the present embodiment, the distance from the final surfaceof the ocular optical system to the eye is calculated at the time ofcalibration for line-of-sight detection. Thus, once a user-specificdistance has been obtained and recorded, from the next and subsequenttimes, it is possible to easily read out the display area optimized forthe user only by selecting the calibration data. Therefore, the user canquickly transition to image capturing.

Second Embodiment

Hereinafter, a second embodiment will be described with reference toFIGS. 13A and 13B.

In the first embodiment, when changing the display area of the display,both the OSD display and the live view display are changed. In contrast,in the second embodiment, when changing the display area, only the liveview display is changed and the OSD display is maintained at a constant.

FIG. 13A illustrates a case where the eye is close to the ocular opticalsystem. FIG. 13B illustrates a case where the eye is far from the ocularoptical system.

In this embodiment, the OSD display is constant regardless of thedistance of the eye from the camera and only the display area of thelive view display is changed. Thus, compared with the configuration inwhich both the OSD display and the live view display are changed, it ispossible to prevent the fonts of characters indicating information, suchas camera settings and the remaining amount of battery, from becomingsmall and difficult to see, while ensuring visibility of the live viewdisplay.

Third Embodiment

Hereinafter, a third embodiment will be described with reference to FIG.14 .

In the first embodiment, the distance 32 from the final surface of theocular optical system 16 to the eye is calculated at the time ofcalibration for line-of-sight detection. In contrast, in the thirdembodiment, the distance is calculated after the approach of the eye isdetected.

In step S1401 of FIG. 14 , the CPU 3 detects the approach of the eyeusing the eye-approach detection unit 208.

In step S1402, the CPU 3 determines whether or not the distance from thefinal surface of the ocular optical system 16 to the eye has stabilized.Here, the CPU 3 determines whether or not the amount of reflected lightreceived by the eye-approach detection sensor 50 is stable exceeding aneye-approach determination threshold. If the amount of reflected lightis stable exceeding the eye-approach determination threshold, it isdetermined that the distance has stabilized, and the CPU 3 advances theprocess to step S1403. On the other hand, if the amount of reflectedlight has not exceeded the eye-approach determination threshold or isnot stable, the CPU 3 determines that the distance is not stable andrepeats the process of step S1402 until the distance stabilizes.

In step S1403, the CPU 3 calculates the distance from the final surfaceof the ocular optical system 16 to the eye.

In step S1404, the CPU 3 determines the display area of the displaybased on the distance calculated in step S1403.

In step S1404, the CPU 3 turns the display on based on the display areadetermined in step S1405.

In the present embodiment, the distance from the final surface of theocular optical system to the eye is calculated after it is detected thatthe user's eye has approached the camera. Thus, since the distance iscalculated for each instance of the approach of the eye and the displayarea is changed, display can be performed with the display areaoptimized for the way the user is looking into the viewfinder at eachinstance.

Fourth Embodiment

A fourth embodiment will be described below with reference to FIG. 15 .

In the first embodiment, the distance 32 from the final surface of theocular optical system 16 to the eye is calculated at the time ofcalibration for line-of-sight detection. In contrast, in the fourthembodiment, the distance is calculated when a press of a button forchanging the display area of the display is detected. In the presentembodiment, it is assumed that the front surface of the camera body 1Bis provided with the button for changing the display area of thedisplay.

In step S1501 of FIG. 15 , the CPU 3 detects a press of the button forchanging the display area of the display.

In step S1502, the CPU 3 calculates the distance 32 from the finalsurface of the ocular optical system 16 to the eye.

In step S1503, the CPU 3 determines the display area of the displaybased on the distance calculated in step S1502.

In step S1504, the CPU 3 changes the display area of the display basedon the display area determined in step S1503.

In the present embodiment, when a press of the button for changing thedisplay area of the display is detected, the distance from the finalsurface of the ocular optical system to the eye is calculated. Thus, itis possible to calculate the distance again and perform display with anappropriate display area when the user wishes to change the displayarea.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc(BD)TM), a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2021-181436, filed Nov. 5, 2021, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An electronic device comprising: a displayconfigured to display an image; an ocular optical system configured toview the display; and at least one processor or circuit configured tofunction as a control unit configured to control the display so as tochange a display area of the display based on a distance from the ocularoptical system to an eye of a user looking into the ocular opticalsystem.
 2. The electronic device according to claim 1, wherein in a casewhere the distance from the ocular optical system to the eye is a firstdistance, the control unit sets the display area to a first size, and ina case where the distance from the ocular optical system to the eye is asecond distance that is longer than the first distance, the control unitsets the display area to a second size that is smaller than the firstsize.
 3. The electronic device according to claim 1, wherein the controlunit changes the display area in a stepwise manner in accordance withthe distance from the ocular optical system to the eye.
 4. Theelectronic device according to claim 1, wherein the at least oneprocessor or circuit is configured to further function as an obtainingunit configured to obtain the distance from the ocular optical system tothe eye.
 5. The electronic device according to claim 4, wherein theobtaining unit obtains the distance from the ocular optical system tothe eye, using an image of a reflection in a cornea of the eye.
 6. Theelectronic device according to claim 5, wherein using the image of thereflection in a cornea of the eye comprises determining the coordinatesof feature points of the eye in the image of the reflection in a cornea.7. The electronic device according to claim 4, wherein the at least oneprocessor or circuit is configured to further function as: aline-of-sight detection unit configured to detect a line of sight,wherein the obtaining unit obtains the distance from the ocular opticalsystem to the eye at a time of calibration of the line-of-sightdetection unit.
 8. The electronic device according to claim 7, whereinthe line-of-sight detection unit changes a display of a target that isused for calibration based on the distance from the ocular opticalsystem to the eye.
 9. The electronic device according to claim 4,wherein in a case where it is detected that a user's eye has approachedthe ocular optical system, the obtaining unit obtains a distance fromthe ocular optical system to the eye.
 10. The electronic deviceaccording to claim 4, wherein in a case where the display area of thedisplay has been changed, the obtaining unit obtains the distance fromthe ocular optical system to the eye.
 11. The electronic deviceaccording to claim 1, wherein in a case of changing the display area ofthe display, the control unit changes the display area of both an OSDdisplay and a live-view display.
 12. The electronic device according toclaim 1, wherein in a case of changing the display area of the display,the control unit changes only a live-view display and maintains adisplay area of an OSD display to be constant.
 13. The electronic deviceaccording to claim 1, further comprising: an image capture deviceconfigured to capture an image.
 14. A method of controlling anelectronic device including a display configured to display an image andan ocular optical system configured to view the display, the methodcomprising: controlling the display so as to change a display area ofthe display based on a distance from the ocular optical system to an eyeof a user looking into the ocular optical system.
 15. A non-transitorycomputer-readable storage medium storing a program for causing acomputer to execute the method of controlling an electronic deviceincluding a display configured to display an image and an ocular opticalsystem configured to view the display, the method comprising:controlling the display so as to change a display area of the displaybased on a distance from the ocular optical system to an eye of a userlooking into the ocular optical system.